Cleaning Data

  1. .mat to .csv file for fasten analysis process
  1. filter disposition file for artists/menu task

Artist/ Menu Task Detection

Find Engagement point

We used AMOC method in changepoint library with assumption that there is single potential changepoint from time when task was available to participant to when task was correctly completed.

We determined changepoint with mean and variance of speed and lane deviation. Then, selected minimum timeframe (faster changepoint detection result) as predicted engagement point for artist task.

  • no predicted engagement point was after the participant was incorrect

Example

exception Two simulations had one artist/menu task engaged (score didn’t change when task was available): Participant 123 YP (20140111102408.csv) and Participant 129 XM (20140104090136.csv)

Define Control vs Experiment Group

Start of experiment group: using task availabe time:

using predictedengagement time:

Visuals

We found that total time (from task available to complete) is longer as participant experience more task.

## # A tibble: 3 x 6
##   eventNum   min    Q1 median    Q3   max
##   <fct>    <int> <dbl>  <dbl> <dbl> <int>
## 1 1           86  180.   282.  602.   619
## 2 2          161  366.   512.  603    612
## 3 3          245  450.   603   604    617

Standard deviation of lane deviation by dosage group and secondary task engagement:

Conclusions:

Boxplot version:

Conclusions:

Paired (trt/control) version:

Conclusions:

  • Within each experiment condition individual subject performance is worse when engaged in secondary task

Within Subject Performance Differences by Condition

Conclusions:

  • Some (but not all) of the instances where performance was better while engaged in the secondary task had slower speeds during the task
  • Difficult to gauge condition effects from this plot

Tables

Artist/Menu task segments available for analysis for each of the 19 subjects:

XM XP YM YP ZM ZP Sum
Subject ID 3 3 3 3 3 3 3 18
Subject ID 7 3 3 3 3 3 3 18
Subject ID 10 3 3 3 3 3 3 18
Subject ID 15 3 3 3 3 3 3 18
Subject ID 17 3 3 3 3 3 3 18
Subject ID 18 3 3 3 3 3 3 18
Subject ID 21 3 3 3 3 3 3 18
Subject ID 25 3 3 3 3 3 3 18
Subject ID 26 3 3 3 3 3 3 18
Subject ID 29 3 3 3 3 3 3 18
Subject ID 31 3 3 3 3 3 3 18
Subject ID 32 3 3 3 3 3 3 18
Subject ID 34 3 3 3 3 3 3 18
Subject ID 35 3 3 3 3 3 3 18
Subject ID 104 3 3 3 3 3 3 18
Subject ID 113 3 3 3 3 3 3 18
Subject ID 120 3 3 3 3 3 3 18
Subject ID 123 3 3 3 2 3 3 17
Subject ID 129 2 3 3 3 3 3 17
Sum 56 57 57 56 57 57 340

Conclusions:

Marginal Statistics

Marginally, there is a statistically significant effect of secondary task engagement (for most Dose/Event combos) on standard deviation of lane deviation (p-values are from paired t-tests within subject/event):

DosingLevel EventID mean_SDL sd_SDL n p_value signif
XM 1 -0.1079823 0.6653311 19 0.47929
XM 2 0.3852318 1.0004674 18 0.10234
XM 3 0.6290988 0.9803059 19 0.00515 **
XP 1 0.3818299 0.8399112 19 0.04753 *
XP 2 0.1759859 1.4630276 19 0.60005
XP 3 0.4446652 0.8482441 19 0.02231 *
YM 1 0.2406506 0.9370421 19 0.26295
YM 2 -0.1298222 1.3342822 19 0.67149
YM 3 0.4220633 0.7426300 19 0.01324 *
YP 1 0.4575804 0.8007496 19 0.01274 *
YP 2 0.5653850 1.1621832 18 0.03902 *
YP 3 0.3885585 0.7604960 19 0.02594 *
ZM 1 0.2998543 1.0046808 19 0.19328
ZM 2 0.0252731 1.0177404 19 0.91380
ZM 3 0.6257512 0.8121137 19 0.00078 ***
ZP 1 0.4451658 1.0555926 19 0.06603 .
ZP 2 0.5386620 1.0042601 19 0.01939 *
ZP 3 0.4225396 0.6069259 19 0.00241 **

Models

Model 1 - Base Model

\[y_{ij} = \beta_0 + \gamma_i + \beta_1 SecTask_{ij} + \beta_2 DoseXM_{ij} + \beta_3 DoseYM_{ij} + \ldots + \epsilon_{ij}\]

  • \(y_{ij}\) = Standard Deviation of lane deviation for subject \(i\) at dosing level \(j\)
  • \(\gamma_i\) = subject specific random intercept
  • \(SecTask_{ij}\) = dummy variable indicating observation measures performance during secondary task (1 = sec task, 0 = control segment)
  • \(DoseAA_{ij}\) = dummy variable for dose = “AA” (1 = specified dose, 0 = reference group)
    • In our models we will re-define the reference group such that all dose effects are positive
d3cp$Dose_Grp <- relevel(d3cp$DosingLevel, ref = "ZP")
fit <- lmer(data = d3cp, SD.Lane.Deviation ~ Dose_Grp +  Experiment  + (1 | ID))
fit_sum <- summary(fit)
kable(round(fit_sum$coefficients, 4))
Estimate Std. Error df t value Pr(>|t|)
(Intercept) 0.4598 0.0853 171.3512 5.3935 0.0000
Dose_GrpXM 0.0154 0.1012 655.1849 0.1522 0.8790
Dose_GrpXP 0.0756 0.1007 654.9280 0.7502 0.4534
Dose_GrpYM 0.0873 0.1007 654.9280 0.8665 0.3865
Dose_GrpYP 0.0517 0.1012 655.1849 0.5109 0.6096
Dose_GrpZM 0.0454 0.1007 654.9280 0.4507 0.6524
Experiment 0.3443 0.0583 654.9280 5.9029 0.0000
kable(round(anova(fit), 4))
Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Dose_Grp 0.6434 0.1287 5 655.0996 0.2226 0.9528
Experiment 20.1476 20.1476 1 654.9280 34.8437 0.0000

Conclusions:

  • Significant effect of secondary task engagement on standard deviation of lane deviation
  • No effect of experimental condition

  • A few problems with this model:
    • Doesn’t adjust for differences in speed
    • Skewed distribution of SD.Lane.Deviation
    • Treats all 3 secondary tasks as the same
    • It also isn’t looking for effect modification by Dose_Grp

Model 2 - Log-transformed outcome, control for eventNum and Avg.Speed

fit <- lmer(data = d3cp, log2(SD.Lane.Deviation) ~ relevel(DosingLevel, ref = "XM") + 
              factor(eventNum)  + Avg.Speed + Experiment + (1 | ID))
kable(round(summary(fit)$coefficients, 4))
Estimate Std. Error df t value Pr(>|t|)
(Intercept) -3.9636 0.5686 370.8770 -6.9712 0.0000
relevel(DosingLevel, ref = “XM”)XP 0.1522 0.1954 652.9692 0.7789 0.4363
relevel(DosingLevel, ref = “XM”)YM 0.2630 0.1976 657.7072 1.3311 0.1836
relevel(DosingLevel, ref = “XM”)YP 0.1300 0.1964 653.7328 0.6617 0.5084
relevel(DosingLevel, ref = “XM”)ZM 0.3465 0.1972 657.0216 1.7573 0.0793
relevel(DosingLevel, ref = “XM”)ZP 0.0550 0.1969 656.3927 0.2792 0.7802
factor(eventNum)2 0.4422 0.1426 664.1713 3.1003 0.0020
factor(eventNum)3 0.3166 0.1417 662.7813 2.2341 0.0258
Avg.Speed 0.0288 0.0087 402.0684 3.3167 0.0010
Experiment 0.7464 0.1124 652.2426 6.6397 0.0000
kable(round(anova(fit), 4))
Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
relevel(DosingLevel, ref = “XM”) 9.1783 1.8357 5 655.6375 0.8545 0.5115
factor(eventNum) 21.6956 10.8478 2 659.0886 5.0497 0.0067
Avg.Speed 23.6309 23.6309 1 402.0684 11.0003 0.0010
Experiment 94.7050 94.7050 1 652.2426 44.0854 0.0000

Conclusions:

  • Secondary task engagement, speed, and eventID are all important predictors of standard deviation of lane deviation
  • Experimental condition is not significant when controlling for these other predictors

Model 3 - Log-transformed outcome, control for eventNum and Avg.Speed, test for effect modification

  • Does Dose_Grp modify the performance deteroration seen during secondary tasks?
fit <- lmer(data = d3cp, log2(SD.Lane.Deviation) ~ relevel(DosingLevel, ref = "ZP") + 
              factor(eventNum)  + Avg.Speed + Experiment + 
              relevel(DosingLevel, ref = "ZP"):Experiment + (1 | ID))
kable(round(summary(fit)$coefficients, 4))
Estimate Std. Error df t value Pr(>|t|)
(Intercept) -3.9386 0.5590 387.3314 -7.0460 0.0000
relevel(DosingLevel, ref = “ZP”)XM 0.0083 0.2779 649.4002 0.0297 0.9763
relevel(DosingLevel, ref = “ZP”)XP 0.0782 0.2778 650.6645 0.2814 0.7785
relevel(DosingLevel, ref = “ZP”)YM 0.2091 0.2754 647.3339 0.7593 0.4479
relevel(DosingLevel, ref = “ZP”)YP 0.1210 0.2795 651.8014 0.4329 0.6652
relevel(DosingLevel, ref = “ZP”)ZM 0.4550 0.2754 647.2304 1.6523 0.0990
factor(eventNum)2 0.4430 0.1430 659.1465 3.0971 0.0020
factor(eventNum)3 0.3175 0.1421 657.7581 2.2334 0.0259
Avg.Speed 0.0286 0.0087 398.7192 3.2839 0.0011
Experiment 0.8304 0.2754 647.2321 3.0158 0.0027
relevel(DosingLevel, ref = “ZP”)XM:Experiment -0.1252 0.3911 647.2303 -0.3201 0.7490
relevel(DosingLevel, ref = “ZP”)XP:Experiment 0.0399 0.3894 647.2418 0.1025 0.9184
relevel(DosingLevel, ref = “ZP”)YM:Experiment -0.0024 0.3894 647.2402 -0.0063 0.9950
relevel(DosingLevel, ref = “ZP”)YP:Experiment -0.0902 0.3911 647.2304 -0.2306 0.8177
relevel(DosingLevel, ref = “ZP”)ZM:Experiment -0.3269 0.3894 647.2665 -0.8394 0.4015
kable(round(anova(fit),4))
Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
relevel(DosingLevel, ref = “ZP”) 8.1640 1.6328 5 649.3536 0.7556 0.5821
factor(eventNum) 21.7850 10.8925 2 654.0749 5.0409 0.0067
Avg.Speed 23.3021 23.3021 1 398.7192 10.7838 0.0011
Experiment 94.6682 94.6682 1 647.2411 43.8109 0.0000
relevel(DosingLevel, ref = “ZP”):Experiment 2.5570 0.5114 5 647.2534 0.2367 0.9463

Conclusions:

  • Dose_Grp doesn’t appear to modify the effect of secondary task engagement

Model 4 - Paired Difference as Outcome

fit <- lmer(data = df, diff_SD_lane_pos ~ relevel(DosingLevel, ref = "YM") + factor(EventID) + diff_speed + (1 | SubjectID))
kable(round(summary(fit)$coefficients, 4))
Estimate Std. Error df t value Pr(>|t|)
(Intercept) 0.1210 0.1562 211.2718 0.7743 0.4396
relevel(DosingLevel, ref = “YM”)XM 0.1231 0.1778 312.8555 0.6922 0.4893
relevel(DosingLevel, ref = “YM”)XP 0.1565 0.1769 312.6196 0.8849 0.3769
relevel(DosingLevel, ref = “YM”)YP 0.2848 0.1778 312.7221 1.6015 0.1103
relevel(DosingLevel, ref = “YM”)ZM 0.1390 0.1776 312.9546 0.7822 0.4347
relevel(DosingLevel, ref = “YM”)ZP 0.2910 0.1770 312.6593 1.6443 0.1011
factor(EventID)2 -0.0315 0.1305 315.2556 -0.2413 0.8095
factor(EventID)3 0.2020 0.1279 314.3777 1.5790 0.1153
diff_speed -0.0004 0.0178 330.3307 -0.0235 0.9813
kable(round(anova(fit),4))
Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
relevel(DosingLevel, ref = “YM”) 3.3992 0.6798 5 312.8148 0.7622 0.5776
factor(EventID) 3.6341 1.8170 2 314.1180 2.0372 0.1321
diff_speed 0.0005 0.0005 1 330.3307 0.0006 0.9813

Conclusions:

  • Little substantive difference analyzing the data this way

Model 5 - Avg Speed as Outcome

fit <- lmer(data = d3cp, Avg.Speed ~ relevel(DosingLevel, ref = "YP") + 
              factor(eventNum)  + Experiment   +  (1 | ID))
kable(round(summary(fit)$coefficients, 4))
Estimate Std. Error df t value Pr(>|t|)
(Intercept) 63.3018 1.2485 34.3184 50.7011 0.0000
relevel(DosingLevel, ref = “YP”)XM -1.5575 0.8088 653.0613 -1.9257 0.0546
relevel(DosingLevel, ref = “YP”)XP -0.2623 0.8051 653.0257 -0.3258 0.7447
relevel(DosingLevel, ref = “YP”)YM -5.2056 0.8051 653.0257 -6.4659 0.0000
relevel(DosingLevel, ref = “YP”)ZM -4.9229 0.8051 653.0257 -6.1148 0.0000
relevel(DosingLevel, ref = “YP”)ZP -4.6509 0.8051 653.0257 -5.7769 0.0000
factor(eventNum)2 4.2126 0.5693 653.0241 7.3998 0.0000
factor(eventNum)3 4.0525 0.5666 652.9927 7.1522 0.0000
Experiment 0.0939 0.4640 652.9927 0.2025 0.8396
kable(round(anova(fit),4))
Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
relevel(DosingLevel, ref = “YP”) 3347.6698 669.5340 5 653.0148 18.2942 0.0000
factor(eventNum) 2589.6467 1294.8233 2 653.0136 35.3794 0.0000
Experiment 1.5003 1.5003 1 652.9927 0.0410 0.8396

Conclusions:

  • Dosing condition has a highly significant impact on participant speed
  • Secondary task engagement doesn’t predict speed
  • No interaction between dosing and secondary task engagment (model testing this not shown, p > .9)